Bayesian probability

This document introduces the foundations of Bayesian probability theory. The emphasis is on understanding why Bayesian probability theory works, and on realizing that the theory relies, on the one hand, on a very limited number of fundamental properties for information processing, and, on the other hand, on a number of application-dependent and arbitrary choices for decision making. 1 What is Bayesian probability theory? Bayesian probability, is one of the major theoretical and practical frameworks for reasoning and decision making under uncertainty. The historical roots of this theory lie in the late 18th, early 19th century, with Thomas Bayes [2] and Pierre-Simon de Laplace [6]. It was “forgotten” for a long time, and began to be re-appreciated in different application domains, during various periods of the 20th century. Hence, Bayesian probability was never developed as one single, homogeneous piece of scientific activity. So, it should come as no surprise that its concepts, methods and solution practices became known under various names: • the Bayesian approach to uncertainty reasoning. • Bayesianism. • the Bayesian framework. • the Bayesian paradigm. • plausible inference. • Bayesian reasoning. • . . . This course speaks most often of Bayesian probability (theory), mainly because its terminology borrows most from statistical probability theory. The “theory” is much more than just a theory: it’s a systematic way of approaching an application problem in which one is confronted with incomplete knowledge about